Asynchronous and Distributed Data Augmentation for Massive Data Settings

Data augmentation (DA) algorithms are widely used for Bayesian inference due to their simplicity. In massive data settings, however, DA prohibitively slow because they pass through the full in any iteration, imposing serious restrictions on usage despite advantages. Addressing this problem, we develop a framework extending that exploits asynchronous and distributed computing. The extended algorithm is indexed by parameter $r \in (0, 1)$ called Asynchronous Distributed (AD) with original as its parent. Any ADDA starts dividing into $k$ smaller disjoint subsets storing them processes, which could be machines or processors. Every iteration of augments only an $r$-fraction some positive probability leaves remaining $(1-r)$-fraction augmented unchanged. draws obtained using new old data. For many choices $r$, fractional updates lead significant speed-up over parent it reduces version when $r=1$. We show Markov chain Harris ergodic desired stationary distribution under mild conditions algorithm. demonstrate numerical advantages three representative examples corresponding different kinds settings encountered applications. all these examples, our generalization significantly faster than $r$. also establish geometric ergodicity turn yields asymptotically valid standard errors estimates posterior quantities.